To deal with the co-channel interference in Device-to-Device (D2D) communication-empowered cellular networks, the sum rate of D2D links was maximized through joint channel allocation and power control while satisfying the power constraints and the Quality-of-Service (QoS) requirements of cellular links. In order to efficiently solve the mixed-integer non-convex programming problem corresponding to the above resource allocation, the original problem was transformed into a Markov decision process, and a Deep Deterministic Policy Gradient (DDPG) algorithm-based mechanism was proposed. Through offline training, the mapping relationship from the channel state information to the optimal resource allocation policy was directly built up without solving any optimization problems, so it could be deployed in an online fashion. Simulation results show that compared with the exhausting search-based mechanism, the proposed mechanism reduces the computation time by 4 orders of magnitude (99.51%) at the cost of only 9.726% performance loss.